Privacy-Preserving Face Recognition with Learnable Privacy Budgets in Frequency Domain

Face recognition technology has been used in many fields due to its high recognition accuracy, including the face unlocking of mobile devices, community access control systems, and city surveillance. As the current high accuracy is guaranteed by very deep network structures, facial images often need to be transmitted to third-party servers with high computational power for inference. However, facial images visually reveal the user's identity information. In this process, both untrusted service providers and malicious users can significantly increase the risk of a personal privacy breach. Current privacy-preserving approaches to face recognition are often accompanied by many side effects, such as a significant increase in inference time or a noticeable decrease in recognition accuracy. This paper proposes a privacy-preserving face recognition method using differential privacy in the frequency domain. Due to the utilization of differential privacy, it offers a guarantee of privacy in theory. Meanwhile, the loss of accuracy is very slight. This method first converts the original image to the frequency domain and removes the direct component termed DC. Then a privacy budget allocation method can be learned based on the loss of the back-end face recognition network within the differential privacy framework. Finally, it adds the corresponding noise to the frequency domain features. Our method performs very well with several classical face recognition test sets according to the extensive experiments.Comment: ECCV 2022; Code is available at https://github.com/Tencent/TFace/tree/master/recognition/tasks/dctd

DuetFace: Collaborative Privacy-Preserving Face Recognition via Channel Splitting in the Frequency Domain

With the wide application of face recognition systems, there is rising concern that original face images could be exposed to malicious intents and consequently cause personal privacy breaches. This paper presents DuetFace, a novel privacy-preserving face recognition method that employs collaborative inference in the frequency domain. Starting from a counterintuitive discovery that face recognition can achieve surprisingly good performance with only visually indistinguishable high-frequency channels, this method designs a credible split of frequency channels by their cruciality for visualization and operates the server-side model on non-crucial channels. However, the model degrades in its attention to facial features due to the missing visual information. To compensate, the method introduces a plug-in interactive block to allow attention transfer from the client-side by producing a feature mask. The mask is further refined by deriving and overlaying a facial region of interest (ROI). Extensive experiments on multiple datasets validate the effectiveness of the proposed method in protecting face images from undesired visual inspection, reconstruction, and identification while maintaining high task availability and performance. Results show that the proposed method achieves a comparable recognition accuracy and computation cost to the unprotected ArcFace and outperforms the state-of-the-art privacy-preserving methods. The source code is available at https://github.com/Tencent/TFace/tree/master/recognition/tasks/duetface.Comment: Accepted to ACM Multimedia 202

Friend Ranking in Online Games via Pre-training Edge Transformers

Friend recall is an important way to improve Daily Active Users (DAU) in online games. The problem is to generate a proper lost friend ranking list essentially. Traditional friend recall methods focus on rules like friend intimacy or training a classifier for predicting lost players' return probability, but ignore feature information of (active) players and historical friend recall events. In this work, we treat friend recall as a link prediction problem and explore several link prediction methods which can use features of both active and lost players, as well as historical events. Furthermore, we propose a novel Edge Transformer model and pre-train the model via masked auto-encoders. Our method achieves state-of-the-art results in the offline experiments and online A/B Tests of three Tencent games.Comment: Accepted by the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2023

Privacy-Preserving Face Recognition Using Random Frequency Components

The ubiquitous use of face recognition has sparked increasing privacy concerns, as unauthorized access to sensitive face images could compromise the information of individuals. This paper presents an in-depth study of the privacy protection of face images' visual information and against recovery. Drawing on the perceptual disparity between humans and models, we propose to conceal visual information by pruning human-perceivable low-frequency components. For impeding recovery, we first elucidate the seeming paradox between reducing model-exploitable information and retaining high recognition accuracy. Based on recent theoretical insights and our observation on model attention, we propose a solution to the dilemma, by advocating for the training and inference of recognition models on randomly selected frequency components. We distill our findings into a novel privacy-preserving face recognition method, PartialFace. Extensive experiments demonstrate that PartialFace effectively balances privacy protection goals and recognition accuracy. Code is available at: https://github.com/Tencent/TFace.Comment: ICCV 202

Transonic flutter analysis of an AGARD 445.6 wing in the frequency domain using the Euler method

A method based on the Euler equations is proposed for solving transonic flutter problems. The transonic nonlinear flow field with local shock wave/boundary layer interaction is obtained by the Euler/boundary layer equations, and the aerodynamic forces are converted from the time domain to the frequency domain using system identification techniques. The structural dynamic equations in generalized coordinates are adopted for solving structure problems. The method is validated by a flutter boundary prediction of the AGARD 445.6 wing model. The simulation results show that the method presented in this paper is accurate for the prediction of transonic flutter boundary through comparison with experimental data and other simulation results. Furthermore, the present frequency domain method is also much more efficient than the time domain method

Exploration of street space architectural color measurement based on street view big data and deep learning-A case study of Jiefang North Road Street in Tianjin.

Urban space architectural color is the first feature to be perceived in a complex vision beyond shape, texture and material, and plays an important role in the expression of urban territory, humanity and style. However, because of the difficulty of color measurement, the study of architectural color in street space has been difficult to achieve large-scale and fine development. The measurement of architectural color in urban space has received attention from many disciplines. With the development and promotion of information technology, the maturity of street view big data and deep learning technology has provided ideas for the research of street architectural color measurement. Based on this background, this study explores a highly efficient and large-scale method for determining architectural colors in urban space based on deep learning technology and street view big data, with street space architectural colors as the research object. We conducted empirical research in Jiefang North Road, Tianjin. We introduced the SegNet deep learning algorithm to semantically segment the street view images, extract the architectural elements and optimize the edges of the architecture. Based on K-Means clustering model, we identified the colors of the architectural elements in the street view. The accuracy of the building color measurement results was cross-sectionally verified by means of a questionnaire survey. The validation results show that the method is feasible for the study of architectural colors in street space. Finally, the overall coordination, sequence continuity, and primary and secondary hierarchy of architectural colors of Jiefang North Road in Tianjin were analyzed. The results show that the measurement model can realize the intuitive expression of architectural color information, and also can assist designers in the analysis of architectural color in street space with the guidance of color characteristics. The method helps managers, planners and even the general public to summarize the characteristics of color and dig out problems, and is of great significance in the assessment and transformation of the color quality of the street space environment